Skip to content

feat(analytics): posthog audit — remove noise, add 10 new events#3956

Closed
waleedlatif1 wants to merge 5 commits intostagingfrom
analytics/posthog-audit
Closed

feat(analytics): posthog audit — remove noise, add 10 new events#3956
waleedlatif1 wants to merge 5 commits intostagingfrom
analytics/posthog-audit

Conversation

@waleedlatif1
Copy link
Copy Markdown
Collaborator

Re-opening of #3917 — originally merged into staging but lost when the branch was accidentally deleted and recreated from main.

TheodoreSpeaks and others added 5 commits April 3, 2026 15:24
* feat(block): add cloudwatch integration

* Fix bun lock

* Add logger, use execution timeout

* Switch metric dimensions to map style input

* Fix attribute names for dimension map

* Fix import styling

---------

Co-authored-by: Theodore Li <theo@sim.ai>
Remove task_marked_read (fires automatically on every task view).

Add workspace_id to task_message_sent for group analytics.

New events:
- search_result_selected: block/tool/trigger/workflow/table/file/
  knowledge_base/workspace/task/page/docs with query_length
- workflow_imported: count + format (json/zip)
- workflow_exported: count + format (json/zip)
- folder_created / folder_deleted
- logs_filter_applied: status/workflow/folder/trigger/time
- knowledge_base_document_deleted
- scheduled_task_created / scheduled_task_deleted
@cursor
Copy link
Copy Markdown

cursor bot commented Apr 4, 2026

PR Summary

Medium Risk
Adds new CloudWatch tool endpoints that accept AWS credentials and query CloudWatch, increasing security and operational risk if auth/logging is misconfigured. Remaining changes are analytics instrumentation with low functional impact but could affect event volume/PII if properties expand.

Overview
Expands PostHog instrumentation and trims noisy events. Adds new server/client events for folder create/delete, schedule create/delete, knowledge-base document deletion, workflow import/export, logs filter usage, and search result selection (now including workspace_id and query length); removes task_marked_read tracking and stops emitting it from the mothership chat API.

Introduces an AWS CloudWatch integration. Adds a new CloudWatch block plus a set of internally-authenticated API routes and tool configs to query Log Insights, list log groups/streams, fetch log events, list/get metric statistics, and describe alarms, including selector support for log group/stream pickers and new AWS SDK dependencies.

Reviewed by Cursor Bugbot for commit 728fa7b. Configure here.

@vercel
Copy link
Copy Markdown

vercel bot commented Apr 4, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Apr 4, 2026 11:43pm

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps bot commented Apr 4, 2026

Greptile Summary

This PR re-introduces the PostHog analytics audit from the accidentally-deleted branch: it removes the noisy task_marked_read event, adds workspace_id to task_message_sent, and introduces 10 new events (search_result_selected, workflow_imported, workflow_exported, folder_created, folder_deleted, logs_filter_applied, knowledge_base_document_deleted, scheduled_task_created, scheduled_task_deleted, task_renamed/task_marked_unread/task_deleted). It also ships the CloudWatch block with full multi-operation support and cascading log-group/log-stream selectors.

Key highlights:

  • All new events are typed in lib/posthog/events.ts — the single source of truth for the event catalog
  • Analytics calls are placed after the success path in every handler (no phantom events on failure)
  • The CloudWatch block correctly separates type coercions into tools.config.params (post-resolution) rather than tools.config.tool (pre-resolution)
  • The cascading cloudwatch.logGroups → cloudwatch.logStreams selector uses logGroupName in the query key and in SELECTOR_CONTEXT_FIELDS, enabling proper cache invalidation when the log group changes
  • posthog is accessed through a ref in all new event-firing code to avoid stale-closure bugs in useCallback

Confidence Score: 5/5

Safe to merge — no P0 or P1 issues found; all new analytics events are fired after the success path with proper type safety.

The only findings are two P2 cache-key observations for the CloudWatch selectors (incorrect credentials after a key rotation would resolve itself after staleTime anyway, and no data loss occurs) and a minor style inconsistency in ref-sync pattern. None of these block production correctness.

apps/sim/hooks/selectors/registry.ts — the two cloudwatch selector query keys should include a prefix of awsSecretAccessKey to avoid serving stale log-group/stream lists after a secret rotation.

Important Files Changed

Filename Overview
apps/sim/lib/posthog/events.ts Adds 10 new typed event definitions, removes noisy task_marked_read, adds workspace_id to task_message_sent
apps/sim/app/workspace/[workspaceId]/home/home.tsx Adds task_message_sent capture on submit; uses useEffect for posthog ref sync (minor style inconsistency)
apps/sim/app/workspace/[workspaceId]/w/hooks/use-export-workflow.ts Adds workflow_exported event after successful download with correct format/count derivation
apps/sim/app/workspace/[workspaceId]/w/hooks/use-import-workflow.ts Adds workflow_imported event with correct format detection and workflow count
apps/sim/app/workspace/[workspaceId]/logs/components/logs-toolbar/logs-toolbar.tsx Adds logs_filter_applied capture on each filter change including custom date range
apps/sim/app/workspace/[workspaceId]/w/components/sidebar/components/search-modal/search-modal.tsx Adds search_result_selected event across all result-type handlers using deferred search ref for query length
apps/sim/app/api/folders/route.ts Adds folder_created server-side PostHog event after successful folder creation
apps/sim/app/api/folders/[id]/route.ts Adds folder_deleted server-side event after successful folder deletion
apps/sim/app/api/schedules/route.ts Adds scheduled_task_created event on standalone schedule creation
apps/sim/app/api/schedules/[id]/route.ts Adds scheduled_task_deleted event; gracefully handles null workspaceId with empty string fallback
apps/sim/app/api/knowledge/[id]/documents/[documentId]/route.ts Adds knowledge_base_document_deleted event after successful document deletion
apps/sim/app/api/mothership/chats/[chatId]/route.ts Adds task_renamed, task_marked_unread, task_deleted events; removes task_marked_read consistent with events.ts removal
apps/sim/hooks/selectors/registry.ts Adds cloudwatch.logGroups and cloudwatch.logStreams selectors; query key omits awsSecretAccessKey which could serve stale cache after credential rotation
apps/sim/blocks/blocks/cloudwatch.ts CloudWatch block with full multi-operation support, basic/advanced mode file selectors, and proper param coercions in tools.config.params
apps/sim/lib/workflows/subblocks/context.ts Adds AWS credential fields and logGroupName to SELECTOR_CONTEXT_FIELDS to enable cascading selectors

Sequence Diagram

sequenceDiagram
    participant User
    participant UI as React UI
    participant PH as PostHog
    participant API as Next.js API
    participant DB as Database

    User->>UI: Submit message / export / import / filter
    UI->>UI: captureEvent(posthogRef.current, event, props)
    UI->>PH: track event (client-side)

    User->>API: POST /api/folders (create)
    API->>DB: insert workflowFolder
    API->>PH: captureServerEvent('folder_created')
    API-->>User: 200 OK

    User->>API: DELETE /api/folders/:id
    API->>DB: delete folder
    API->>PH: captureServerEvent('folder_deleted')
    API-->>User: 200 OK

    User->>API: POST /api/schedules
    API->>DB: insert workflowSchedule
    API->>PH: captureServerEvent('scheduled_task_created')
    API-->>User: 201 Created

    User->>API: DELETE /api/schedules/:id
    API->>DB: delete workflowSchedule
    API->>PH: captureServerEvent('scheduled_task_deleted')
    API-->>User: 200 OK

    User->>API: DELETE /api/knowledge/:id/documents/:docId
    API->>DB: delete document
    API->>PH: captureServerEvent('knowledge_base_document_deleted')
    API-->>User: 200 OK

    User->>API: PATCH /api/mothership/chats/:chatId
    API->>DB: update copilotChats
    API->>PH: captureServerEvent('task_renamed' or 'task_marked_unread')
    API-->>User: 200 OK
Loading

Comments Outside Diff (1)

  1. apps/sim/app/workspace/[workspaceId]/home/home.tsx, line 206-208 (link)

    P2 Inconsistent posthog ref sync pattern

    All other files in this PR (use-export-workflow.ts, use-import-workflow.ts, logs-toolbar.tsx, search-modal.tsx) use the inline assignment pattern:

    const posthogRef = useRef(posthog)
    posthogRef.current = posthog

    Here a useEffect is used instead. Both are functionally equivalent (the useRef initializer already seeds the correct value on first render), but the inline pattern is the consistent choice in this codebase. Consider replacing the useEffect with a direct inline assignment for consistency:

    (And remove the useEffect block on lines 206–208.)

    Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Reviews (1): Last reviewed commit: "fix(analytics): correct format field log..." | Re-trigger Greptile

Comment on lines +1727 to +1733
],
enabled: ({ context }) =>
Boolean(context.awsAccessKeyId && context.awsSecretAccessKey && context.awsRegion),
fetchList: async ({ context, search }: SelectorQueryArgs) => {
const body = JSON.stringify({
accessKeyId: context.awsAccessKeyId,
secretAccessKey: context.awsSecretAccessKey,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 awsSecretAccessKey absent from query key

The cloudwatch.logGroups (and cloudwatch.logStreams) query key includes awsAccessKeyId and awsRegion but not awsSecretAccessKey. If a user rotates only the secret key while keeping the same access key ID and region, React Query will serve the stale cached list of log groups until SELECTOR_STALE ms elapse — after which the refetch will fail with an AWS auth error. Adding a short hash (e.g., first 8 chars) of the secret to the key ensures immediate invalidation without leaking the raw secret into cache storage.

Suggested change
],
enabled: ({ context }) =>
Boolean(context.awsAccessKeyId && context.awsSecretAccessKey && context.awsRegion),
fetchList: async ({ context, search }: SelectorQueryArgs) => {
const body = JSON.stringify({
accessKeyId: context.awsAccessKeyId,
secretAccessKey: context.awsSecretAccessKey,
getQueryKey: ({ context }: SelectorQueryArgs) => [
'selectors',
'cloudwatch.logGroups',
context.awsAccessKeyId ?? 'none',
context.awsRegion ?? 'none',
context.awsSecretAccessKey ? context.awsSecretAccessKey.slice(0, 8) : 'none',
],

Comment on lines +1762 to +1769
],
enabled: ({ context }) =>
Boolean(
context.awsAccessKeyId &&
context.awsSecretAccessKey &&
context.awsRegion &&
context.logGroupName
),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Same stale-cache issue for cloudwatch.logStreams

The same awsSecretAccessKey-absent query key problem applies here. A short prefix of the secret key should be included for consistent cache invalidation.

Suggested change
],
enabled: ({ context }) =>
Boolean(
context.awsAccessKeyId &&
context.awsSecretAccessKey &&
context.awsRegion &&
context.logGroupName
),
getQueryKey: ({ context }: SelectorQueryArgs) => [
'selectors',
'cloudwatch.logStreams',
context.awsAccessKeyId ?? 'none',
context.awsRegion ?? 'none',
context.logGroupName ?? 'none',
context.awsSecretAccessKey ? context.awsSecretAccessKey.slice(0, 8) : 'none',
],

Copy link
Copy Markdown

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 728fa7b. Configure here.

onOpenChangeRef.current(false)
},
[workspaceId]
)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Four identical search select handlers differ only by string

Low Severity

The handleTaskSelect, handleTableSelect, handleFileSelect, and handleKbSelect useCallback hooks duplicate logic. Each pushes item.href to the router, captures a search_result_selected event, and closes the modal, differing only by the result_type string. This duplication introduces a maintenance risk.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 728fa7b. Configure here.

@waleedlatif1 waleedlatif1 deleted the analytics/posthog-audit branch April 5, 2026 01:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants